This invention relates to video compression.
In an important example, the present invention concerns itself with MPEG-2 which is an emerging standard for the digital compression of video signals. As is well known, MPEG-2 is a development of MPEG-1 which was directed towards low bit rate storage applications such as CD-ROM and had no interlace capability. Without specifying the manner in which a video signal is coded, MPEG-2 defines a bitstream syntax and a set of rules for a decoder to regenerate the picture.
One object of the present invention, in certain of its forms, is to provide improved MPEG-2, or other compression, encoders and decoders.
In certain of its aspects, this invention is concerned more particularly with the performance of a signal transmission chain rather than the performance of a single compression coder or decoder, or indeed of a transmission coder and decoder pair (codec).
In a signal transmission chain, several codecs might be connected in cascade with switching, re-multiplexing or other processing operations performed between each codec. In addition, the start (or indeed, any intermediate point of the chain) may involve some pre-processing such as noise reduction, and the end of the chain (or again, an intermediate point) may involve some post-processing such as display conversion.
In a conventional signal chain, each codec pair operates in isolation from other codecs and from the intervening signal processes. This can lead to inefficiency and loss of performance; inefficiency because each encoder is obliged to recalculate all its coding parameters such as motion vectors, and loss of performance because impairments introduced at an earlier point in the chain might unknowingly be re-coded as picture information.
It is an object of one form of the present invention to provide an improved signal transmission chain which removes or reduces this inefficiency and loss of performance.
Accordingly, the present invention consists, in one aspect, in a signal transmission chain comprising at least one compression coder adapted to receive a signal and to generate therefrom a compressed or partially compressed coded signal and at least one compression decoder adapted to receive a compressed or partially compressed coded signal and to generate therefrom a decoded or partially decoded signal, characterised in that an information bus extends from a decoder or to a coder in the chain, the information bus carrying information relating to an earlier coding operation for use in a later signal process.
Suitably, the information bus extends from a decoder to a later coder in the chain.
In the preferred example, the decoded signal is a picture signal.
In one form of the invention, the information bus is used in a post-processing operation such as display upconversion. Similarly, in some arrangements according to the invention, an information bus is generated in a pre-processing operation such as motion compensated noise reduction. Within a signal processing chain, the information bus may additionally be of assistance within signal processes other than encoding/decoding, an example here being motion compensated standards conversion.
This invention is not restricted to any particular compression technology but MPEG-2 will be taken as an example. It is known within MPEG-2 that a compressed or coded picture signal within a codec pair is usefully accompanied by side chain information. This side chain information may be similar to information which the present invention supplies on the information bus. Indeed, it will be convenient to generate the information bus from the side chain. It should be stressed, however, that the invention is providing something very different from what has previously been proposed under MPEG-2. The existence of a side chain within a codec pair cannot cure the inefficiency and loss of performance which has been described as arising between one codec pair and another in a complete signal transmission chain. It is the proposal according to the present invention in which the output of a decoder (usually a full bandwidth picture signal, but sometimes a partially decoded signal) which is accompanied by an information bus, so that subsequent encoding or other processing in later portions of the transmission chain can make use of the information about earlier encoding.
The present invention, in a further aspect, recognizes that certain video processes hitherto conducted by first decoding a compressed signal to a picture, can very usefully be conducted without leaving the compressed domain. One example is bit rate conversion where, for example, a compressed signal at 6 Mbits/s must be converted for transmission along a channel capable of supporting only 4 Mbit/s. Decoding the 6 Mbit/s signal and re-encoding at 4 Mbit/s, runs the risk of introducing fresh errors and is inefficient. It is an object of this invention to provide an improved method of processing which overcomes this problem.
Accordingly, the present invention consists, in a further aspect, in a method of processing a compressed signal, comprising the steps of variable length decoding the signal; inverse quantising such decoded signal to produce a coefficient stream; re-quantising the coefficient stream optionally at a new bit rate and variable length coding the re-quantised, coefficient stream.
This aspect of the invention will find applications outside bit rate conversion. It will for example be necessary in some applications to combine compressed signals. This will arise in relation to the so-called SNR profile of MPEG-2, which will now be described.
There are defined within MPEG-2, different profiles and levels which arexe2x80x94broadly speakingxe2x80x94directed toward different video applications. Thus, for example, xe2x80x9cMain Profilexe2x80x9d at xe2x80x9cMain Levelxe2x80x9d MPEG-2 provides only for standard definition television whereas the xe2x80x9cSpatially Scalable Profilexe2x80x9d at the same level can also accommodate high definition display.
Certain profiles within MPEG-2 employ the concept of xe2x80x9clayeringxe2x80x9d. This intended to enable decoders of different sophistication to operate upon the same coded signal. A basic decoder might make use of only the lowest layer of information in the coded signal; a more advanced decoder would make use of a higher layer or layers.
There is believed to be a need within MPEG-2 to provide for a coded signal which is resilient in the face of less than optimal transmission channels. By this is meant a coded signal which contains sufficient information for the regeneration of a high quality picture, but in a form which can still sensibly be decoded if a portion of the information is lost, albeit to produce a picture of reduced quality. In this context, the concept of layers is again employed with a basic layer of information being routed at all times through the highest priority and most reliable path in the transmission channel. One or more enhancement layers, which are not essential for the regeneration of a picture but which improve performance, are sent along lower priority paths. One profile operating in this manner under MPEG-2 is the so-called SNR Profile.
An SNR-Profile MPEG-2 coder will output two coded streams; a lower layer containing the most significant digits of the DCT coefficients and an enhancement layer containing the least significant digits, in both cases quantised and variable length coded. The SNR Profile signal is of course not compatible with a Main Profile decoder, unless the enhancement layer is discarded.
Already, Main Profile decoders are available relatively cheaply in standard VLSI form; this will not be the case for SNR Profile decoders for some time. Accordingly, there would in a number of applications be economic advantage in converting an SNR Profile signal in two streams, to a single stream Main Profile signal. The trivial example of decoding from one profile to a picture and recoding that picture at the new profile, is obviously unhelpful.
It is one object of this further aspect of the invention to provide for the conversion of an SNR Profile MPEG-2 signal in two streams into a Main Profile signal with the minimum of processing and with minimal loss of information.
It is a further object of this aspect of the invention to provide for elegant and straightforward processing of an MPEG-2 or other compressed signal.
Accordingly, the present invention consists, in a further aspect, in a method of combining a plurality of compressed signals, comprising the steps of variable length decoding each compressed signal; inverse quantising each such decoded signal to produce a coefficient stream; combining the coefficient streams to form a combined coefficient stream; re-quantising the combined coefficient stream and variable length coding the re-quantised, combined coefficient stream.
Preferably, the compressed signals comprise at least two layers of An SNR profile MPEG-2 signal.
The present invention recognises that there exists a novel, intermediate domain in which different layers (such as the lower and enhancement layers in the SNR profile) are readily combinable. Once synchronicity has been ensured, for example, it will usually be possible for the two streams to be simply added. This intermediate domain will be referred to as the coefficient domain.
The present invention will have application well beyond bit rate conversion and the conversion of dual SNR profile streams to main profile format. It will be possible to perform other useful processing within the coefficient domain. Thus, the invention will find application in re-multiplexing. It will permitxe2x80x94for examplexe2x80x94the de-multiplexing of a group of channels, the addition of a further channel in synchronism, and re-multiplexingxe2x80x94all without returning to the level of pixels. Establishing sychronocity between two slightly out of sync bit streams, will sometimes be an end in itself, again readily accomplished in the coefficient domain, following the present invention.
The formation of an information bus for MPEG-2 has already been proposed. Information from the variable length decoding according to the present invention can conveniently feed an information bus generator. This same information can in accordance with the present invention, be supplied to a microcontroller or microprocessor controlling the re-quantisation step to maximise re-coding efficiency and optimise buffer occupancy. Indeed, the ability to extract coding information from an incoming MPEG stream, without decoding to pixels, is itself an important advantage offered by the present invention.